Skip to content

ci: Split up Test AI workflow #5148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Nov 26, 2025
Merged

ci: Split up Test AI workflow #5148

merged 2 commits into from
Nov 26, 2025
+313 −37

Conversation

@alexander-alderman-webb
Copy link
Contributor

@alexander-alderman-webb alexander-alderman-webb commented Nov 25, 2025

Description

The AI test suite takes the longest, so splitting the workflow up reduces the time for CI to complete.

Issues

Reminders

@alexander-alderman-webb alexander-alderman-webb requested a review from a team as a code owner November 25, 2025 16:25
cursor[bot]
cursor bot reviewed Nov 25, 2025
View reviewed changes
.github/workflows/test-integrations-agents.yml
strategy:
fail-fast: false
matrix:
python-version: ["3.10","3.11","3.12","3.13","3.14","3.14t"]
Copy link

@cursor cursor bot Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Python version matrix includes unsupported combinations

The workflow Python version matrices include versions unsupported by some frameworks in each group. The generator script unions all Python versions across grouped frameworks, causing tests to fail for nonexistent tox environments. For example, pydantic_ai lacks Python 3.11, 3.14, and 3.14t in tox.ini, yet the Agents workflow will attempt py3.11-pydantic_ai. Similarly, langgraph lacks 3.11, and mcp lacks 3.14 and 3.14t in their respective workflows.

Additional Locations (2)

Fix in Cursor Fix in Web

@codecov
Copy link

codecov bot commented Nov 25, 2025 •
edited
Loading

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
28630 1 28629 2319
View the top 1 failed test(s) by shortest run time 
tests.integrations.ray.test_ray::test_errors_in_ray_actors
Stack Traces | 0.106s run time 
.../integrations/ray/test_ray.py:229: in test_errors_in_ray_actors
    ray.init(
.tox/py3.9-ray-v2.7.2/lib/python3.9.../ray/_private/client_mode_hook.py:103: in wrapper
    return func(*args, **kwargs)
.tox/py3.9-ray-v2.7.2/lib/python3.9.../ray/_private/worker.py:1536: in init
    _global_node = ray._private.node.Node(
.tox/py3.9-ray-v2.7.2/lib/python3.9.../ray/_private/node.py:303: in __init__
    self._ray_params.update_pre_selected_port()
.tox/py3.9-ray-v2.7.2/lib/python3.9.../ray/_private/parameter.py:348: in update_pre_selected_port
    raise ValueError(
E   ValueError: Ray component dashboard_agent_http is trying to use a port number 52365 that is used by other components.
E   Port information: {'gcs': 'random', 'object_manager': 'random', 'node_manager': 'random', 'gcs_server': 60768, 'client_server': 'random', 'dashboard': 'random', 'dashboard_agent_grpc': 52365, 'dashboard_agent_http': 52365, 'dashboard_grpc': 'random', 'runtime_env_agent': 64941, 'metrics_export': 57935, 'redis_shards': 'random', 'worker_ports': 'random'}
E   If you allocate ports, please make sure the same port is not used by multiple components.

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

sentrivana
sentrivana approved these changes Nov 26, 2025
View reviewed changes
Copy link
Contributor

@sentrivana sentrivana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏🏻

@alexander-alderman-webb alexander-alderman-webb merged commit 7b7ea33 into master Nov 26, 2025
155 checks passed
@alexander-alderman-webb alexander-alderman-webb deleted the webb/split-ai-test-suite branch November 26, 2025 08:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cursor cursor[bot] cursor[bot] left review comments

@sentrivana sentrivana sentrivana approved these changes

Assignees

No one assigned

Labels

Changelog: Internal

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@alexander-alderman-webb @sentrivana